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ABSTRACT 

This guide defines a methodology for program 
monitoring and discusses positive results achieved in Pennsylvania by 
implementing Instrument-based Program Monitoring (IPM) in the field 
of child day care services. IPM uses checklists or special 
questionnaires to structure monitoring interviews and site visits. 
This method differs substantially from the more common approach to 
monitoring: narrative site visit reports. An introduction on the 
nature and purpose of the guide is followed by three substantive 
sections, beginning in section II with a discussion on how IPM 
improves day care monitoring. Specific comments under this heading 
include descriptions of how IPM is performed, objectives and 
characteristics of the IPM approach, and positive outcomes of using 
IPM in Pennsylvania. Section III examines the effectiveness of 
Pennsylvania's day care monitoring system, known as the Child 
Development Program Evaluation (CDPE) . Particular comments are 
directed toward general characteristics of the CDPE system; CDPE 
system components; types and level of information generated 
(including provider, regional, and statewide information); and the 
implementation process. Concluding material in section IV provides 
guidelines for determining a state's need for an IPM system. Five 
steps in conducting an adequate assessment are described, and 
necessary elements of an effective implementation plan are briefly 
outlined. (RH) 
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. INTRODUCTION 

Purpose of the Guide 

* Instniment-based program monitoring (IPM) 

is an innovative approach to slate management of 
human services. The advantages of the approach 
arc panicularly relevant as states reassess their role 
in day care and other services in the current volatile 
fiscal environment. These advantages include: 

• cost savings to states in financing 
services; 

• better state allocation of resources; 

• improved information for policy 
dedsions;and 

^ enhanced quality of programs. 

This Guide defines a methodology for monitor- 
ing that uses an instrument to collect data and 
i discusses positive results that Pennsylvania has 
j achieved by implementing IPM in the field of child 
day care services. Although IPM has been 
developed and applied in day care, there is great 
potential for extending IPM to other human 
services. 

The Guide provides sufficient information for 
most states to decide whether it is worthwhile to ex- 
plore the feasibility of adopting the IPM approach. 
It also presents practical considerations for im- 
plementing such an approach to monitoring. 

For Whom Is the Guide Intended? 

This Guide is the first of three volumes of the 
complete documentation of Pennsylvania's 
monitoring system that embodies the IPM ap- 
proach. It provides an overview of the approach 
and highlights aspects that are important to state 
government officials responsible for top level 
policymaking in day care and other human services. 
These officials include: top executives in state 
human service organizations; legislators and 
legislative staff mcml^rs who must formulate 
policy concerning the objectives and system -wide 
costs and benefits of human services programs; 
state budget staff; and managers with responsibility 
for designing information systems to support 
human services management. 

What Is An Instrument-Based Program 
Monitoring System? 

Instrument-based program monitoring systems 
incorporate three important characteristics: 

• Instrument-Based. IPM utilizes 
checklists or special kinds of question- 



naires to structure monitoring inter- 
views and site visits. Highly specific 
questions incorporate pertinent state 
regulations in a simplified format that is 
easily completed and shows both pro- 
viders and state agencies just how well 
the provider is complying with re- 
quirements. An integral feature of IPM 
is a system of assigning weights to the 
questions or items so that scores reflect 
the relative importance of the regula- 
tions, Pennsylvania's system, for exam- 
ple, includes assigning weights based on 
the degree of risk to children while they 
are in day care. 

• Program Monitoring. In this Guide, 
monitoring is a broad term describing 
the management process of reviewing 
and controlling the delivery of program 
seric« according to predetermined 
criteria, with the intention of taking cor- 
rective action to assure and increase pro- 
gram quality and managonent efficien- 
cy. As such, monitoring encompasses a 
continuum of manag^ent activities, 
from licensing, contract compliance 
reviews and program quality 
assessments to corrective actions, 
technical assistance, and training. 

• Systems Approach. A system is a group 
of related and coordinated procedures 
that are used to accomplish a given task. 
In this instance, the task is to monitor 
day care providers according to state 
regulations. The Guide permits integra- 
tion of data coll^ed through monitor- 
ing with financial information and 
subsequent corrective action. 

An IPM system differs substantially from the 
more common approach to monitoring: narrative 
site visit reports. Typically, narrative reports 
describe a site visit to each provider and summarize 
not only observations but also interpretive and 
evaluative comments about the monitor's findings. 
These repKjrts are useful if the monitor is well train- 
ed and adept at reporting. Even the best monitor, 
however, has biases that are reflected in both the 
selection of topics covered in a report and the 
degree to which the topics are covered. Site visit 
reports are often difficult to summarize succinctly 
for policymakers and even more difficult to use in 
comparing different providers or describing the 
general trends in day care in the state. Further, they 
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often take longer to prepare than questionnaire 
summary reports, and they may be more difficult to 
use as evidence in court if legal action against a 
non<ompIiant provider is necessary. 

Several states currently use instrument -based 
systems for licensing day care providers. In Penn- 
sylvania, licensing standards include both re- 
quirements for protecting the health and safety of 
children and requirements designed to promote 
child development. Systems like Pennsylvania's in- 
clude reviews of program quality and they provide 
a basis for slates to make fundamental judgments 
about the relative child development benefits to be 
gained from particular levels of funding. Such 
judgments are increasingly important in today's 
world of declining resources, increased community 
demands for services, and a more litigious legal en- 
vironment. 

Overview of the Guide 

The sections of this Guide follow the major 
questions that policymakers might raise about the 



value and applicability of IPM bvsfcms for their 
states: 

• How do instrument-based systems 
improve day care monitormg? 

• How well has Pennsylvania's IPM 
performed? 

• How can I evaluate my state's need 
for an IPM system? 

At the back of the Guide is a list of resources 
containing moie detail about the conceptual foun- 
dations of the system for individuals who want to 
pursue these issues further. 

The Guide includes a series of illustrations (see 
box below) showing how Pennsylvania's top ad- 
ministrators have made use of the reports generated 
by the state's IPM system to evaluate policy ques- 
tions that are of central importance in the manage- 
ment of state human services programs. 



SAMPLt INSTRLMF.NT FOR DAV CAKt MOMTOKIXi 



Portions of Pennsylvania's instrumeni for monitoring day care centers are reproduced below Thr fir« u . u 

CHILDREN'S HEALTH RECORDS 




Do all cuts and exit routes bypass hazardous areas such as boiler rooms and 
kiivhrn conking equipment? 



Are all exits and exit routes unobstructed? 



[)o all exit doors swing in the direction of exit travel' 



Do all exjt doors have only one lo^-kmg or latching device *htch can he unlocked 
from the inside at ail times? 




Do all hallways, stairways (including landings > permit passage of two adi lts 
walking side by side' 
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II. HOW DO INSTRUMENT^ 
BASED SYSTEMS IMPROVE 
DAY CARE MONITORING? 



How Is Instrument-Based Program 
Monitoring Performed? 

lastrumcni-bascd program monitoring relics 
on the use of a detailed questionnaire or ch^klist 
based on explicit stote regulations to determine how 
well a day care provider is me^ng state re- 
quirements. When the assessment and question- 
naire are completed, the results arc scor«i using a 
common scoring manual for all provide of a given 
type (for example, day care centers), and coded for 
entry into an information system, either manual or 
automated. The results are used to determine 
whether the state needs to intervene in order to im- 
prove conditions at spedHc provider sites. The pro- 
viden* scores on the questionnaires are also used to 
support state decisions conconing the granting and 
renewal of licenses and state funding. Aggregate, 
statewide scores can be used as a basis for broad 
policy decisions directed at improving the general 
quality of day care and setting state funding levels* 



Objectives and Characteristics of the IPM 
Approach 

IPM systems are more comprehensive, objec- 
tive, and consistent than the narrative report ap-^ 
proach. They are also easier to read and under- 
stand. They arc ideally suited to achieve the follow- 
ing objectives of a day care monitoring system: 

• Enssre suitable, enfwceabte monitoring 
of day care to meet a d^red tevd of child 
health and safety. Most statw have re- 
quirements concerning the health and safe- 
ty of day care centers. It is essential that 
health and safety standards be dearly 
specified, clearly understood by providers, 
easily evaluated, and consistently enforced 
by the state* 

• Ensure that day care promotes child 
development. Typically, a state is concern- 
ed that children who are served by publicly 
fund^ day care provides (or all providers 
in some states, such as Pennsylvania) 
recdve positive experiences from their day 
care in terms of their individual growth and 
development. 



CAN IPM SYSTDMS INCREASE 
THE EFFECTIVENESS OF STATE MONrTORING EFFORTS? 



STATTWmE CmmiANCE WITH DAY CAfifi RCCULATTONS, 197S-1W 



The most basic QLie^ion that st^ 
day caro wbn^Tistrators most acUrBss 
is whether efforts to ga^ oompiiance 
with state re<^ements are prodi^lng 
desired resute, Pennsylva^'s IWW 
system has provided pericx^ ac- 
cessibto yif o fm a lton to deiTOralrG^ 
that the state has sutetarvf^ 
qro^BBB In ^proving ccrnifrftence 
sma the IPM system {c^M the CTnW 
Devefopment Progfam Evaluation 
system m Pennsyf/an^) was kitnxktC' 
ed^ 1978, 

These ^suHs have created support 
for the {PM system et a0 levets of the 
stste government ^d have been ef- 
fective m persuade pt^ymakera to 
maintain current day care monilormg 
efforts. 
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• Provide for efricienf and cosl-effeciive 
funding and monMoring procedures. States 
need to achieve the benefits of monitoring 
as efficiently as possible and at the 
minimum necessary cost. 

• Permif sound policy decision making. 
States are concerned that their funds are 
spent in a way that ensures the best possible 
day care services. To address this concern, 
stale policymakers need consistent, objec- 
tive, quantifiable indicators of how many 
individuals are being served by day care 
and what funding levels are required lo 
serve these individuals at different levels of 
program quality. Further, policymakers 
require an information base for deciding 
how policy should evolve in the future and 
for developing and maintaining ap- 
propriate legislation, regulations and 
policy guidelines. 

Achieving these objectives requires timely, 
reliable, concise information about a state's day 
care program. An IPM system can provide this 
kind of information and has the following addi- 
tional beneficial charaaeristics: 

• Quantitative and Objective. Using a pro- 
gram monitoring instrument produces 
clear, specific, and objective information 
about programs. The use of highly struc- 
tured questions and records of on-site 
observations minimizes ambiguous results 
and biased observations. Questionnaire 
items lend themselves to quantitative 
analyses and produce readily summarized 
and easily interpreted data that are of value 
fo policymakers. 

•Fiasfly Administered and Consisieni. The 

questionnaires can be administered by 
monitors with varying academic 
backgrounds who have been trained to 
gather information quickly and with a 
minimum of interference with the ongo 
:ng activities of the provider. While much 
of a questionnaire may deal with evaluative 
and programmatic information, responses 
from various providers and monitors are 
hignly a)nsisfcnt. 

•Supporiivi- of Providm. Many provider'. 
>*clc()me the use of such uistrumc-nts 
K-cause rhsry know what specific areas are 



covered and they can structure their pro- 
grams to meet state expectations, involve- 
ment of providers in developing the ques- 
tionnaire funher ensures that the questions 
will be comprehensive without being 
burdensome. The use of questions that ex- 
plicitly cover all regulations, and establish 
a uniform set of requirements, increases 
the providers' perception that the re- 
quirements are equitable, necessary, and 
desirable with respect to improving day 
care. 

• Focus on Results. The design of the ques- 
tionnaire reduces undesirable bias that may 
result from differences in temperament and 
child care philosophy between the provider 
and the state monitor. The IPM focus is 
entirely on the services provided and how 
they affect children. 

• Based on State-of-tfie-Art Child Develop, 
meni Research. The questionnaire items 
can be designed to reflect current 'best 
practice' in day care. In this way, the ques- 
tions may be used to encourage providers 
to experiment with and adopt successful 
approaches that have been shown to be ef- 
fective in research in child development. 
1PM can be used to improve day care, and 
not just regulate it to ensure that minimum 
requirements are met. 

• Easy to .Modify and Improve. The ques- 
tionnaire format with specific and unam- 
biguous responses is easily adapted to 
changing developments in child care prac- 
tices and state policy requirements. State 
requirements can be easily communicated 
to funded providers by incorporating new 
or revised questions on which assessments 
will be made.lt is also easy to incorporate 
any federal requirements that may exist. 



Desirable ResulCs From Using IP.M: 
Pennsylvania's Experience 

The positive features of 1PM that have been 
described above may have particularly bene.^cial 
results for a state, if Pennsylvania's experience is a 
guide. Since Pennsylvania's system was introduced 
m 1978, it has produced the following im- 
provements: 



^ COST-EFFECTIVE FOR A STATE 
TO FUND HIGHER-COST DAY CARE CENTERS? 

As state fimds become mwe so&ce, nmy ^ po^ 
meters im^ confront the potent ^^fen^ between tlw 
of care s^vtces piovkM by stMhmiM prov^iera 
tt^ ttie costs »so^alBd wm ctesir^ qua^ toveto. rt is 
iBMor^sls to expect tfM as (^1^ bnjmvee provfctor costs 
rise, bt^ ft to not ctesr tfi^ (^i^ ^T^^roves fmi^ beycf^ 
^ cos^ lev; . Most are im^to lo i ^ Hw mitw ttet rast 
le^, fmvever, md t^tm decerns tesad on fiscal 
r^cas^ or pc^tteal lobby^. 

Us^ its (PM system, PennsytvOT^ mfclrraaed the <^^ty/cost 
c^^^ton <Srec«y. Easiy accm^^ <SBta frwn ate systmr per- 
fi^lsd an sf^ys^ in^ sN^A^ (^^gfy ^^f^^ at 3 ^cft^ 
We above a partta/^ tovef of coet-per-hoir ^ swice p»x)v<d' 
ed. (See figure below.) 

Thm sm9 tmmf <^^kfy iQ ^ th0 f9V99 of com M ¥ftHch » 
woiM rBiff^Htrs9 tm^lders. The move saved PemsyfienAinear' 
fy $5 ntiibn rAal was tfwt reaipc^ed fo ^mpnove efie dtoy cira 
$>n)^^mm of tower cost providefS (fM nwf stafe s^m<Smtis. 



WAUTY-COST CLUVf: 
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• Cost Retiucfions. By linking the results of 
ihcir IPM to the state's information 
system, officials in Pennsylvania have been 
able to identify high-cost, publicly funded 
providers who offer services onl> marginal- 
ly better in quality than those of lower-cost 
providers- The state was able to set a ceil- 
ing on day care funding that did not 
jeopardize program quality, and used the 
funds that were formerly given to high-cost 
providers to improve services of other pro- 
viders on a targeted basis. The slate saved 
approximately $5 million in day care funds 
while maintaining the quality of day care 
services, and it did so without major 
resistance from the provider groups. 



• Iinproved Prograoi Performance. Pro- 
viders' scores on the monitoring question- 
naire have improved over time as providers 
focus on meeting the state's clearly defined 
expectations, ^ause th^ exf^tattons 
reflect both program quality and basic 
licensing standards, these improved scores 
indicate that the state-funded day care ser- 
vices have become markedly better in a 
short period. 

• Improved Regulatory Qimate* By involv- 
ing provido^ in all stages of the develop- 
ment and improvement of the quct>fi/«n- 
naire, and by using the questionnaiu lo 
create clear expectations of providers, of- 
ficials responsible for day care monitoring 
have reduced the tension and legal conflicts 
that often arise in regulated industries. 
Providers are satisfi^ that the question- 
naires are fair and that they are ad- 
ministered consistently and without special 
treatment given to particular providers. 

e Improved Information for Policy and 
Finamriai Decistons. By linking the results 
from the IPM system to information 
systems that provide financial and 
statistical information on day care, of- 
ficials in Pennsylvania have been able to 
make financial decistons about cost ceilings 
without much of the conflia that usually 
accompanies such decisions. The state is 
also in a strong position to complete the 
implementation of unit resting and com- 
petitive procurement systems in a way that 
explicitly considers program quality. 
Ultimately, the information provided 
could be used to answer such basic policy 
questions as: *'Dms state-funded day care 
really enhance the development of children 
for whom it is provided?** In the meantime, 
policymakers arc receiving concise, quan- 
titative, useful, and timely information to 
support difficult decisions with respect to 
trade-offs among state services in a period 
of tight resources. 

Having described the advantages of the IPM 
approach, it is useful to examine the features of a 
successful, currently operating system to get a more 
concrete idea of the costs and operating 
characteristics. In the next section, Pennsylvania's 
Child Development Program Evaluation System is 
described briefly as a working example of 1PM 
systems. 
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T(9 f^msyhmNa »Mntetrators were ^terestod in determn- 
Irig wtieti^ m« 9ize of a day care cemor, meaai^ ^ mj^^ 
of cNk^ servad, ralaaed to the (^^lyof tfte centos 

tefBsfing reiattsraf^. Average scotm \mrgQ cmtm 
begm to ctoomse dtm^ a point even tNx^ the ym^^t^ m 
cpfflgty. as mea&rad by the sccres, as gr^ m l^ge 

as bt smaS cantors. 

TIA erf sane decrease 01 tfto <^ia% erf flME^Ice tor te^ 
provfctors has id^H^ by several rmmait^ms ^ 
care (Preacott, mM„ 1972). Tlimo c^iar rscwchm 
hypc^^eaiced that as centm bacons very targe, tt^ ad* 
firin^afr;^^ comptoxfty ami afmer site crMa aofTw siTi^ 
the (toy cm Tlieae afr^ »e co n am a A b a ted to tfie 
chidren to the torm of toaa personal attantton and less con* 
actoM ^tontton to itototo in compBance arm. 

P^msyfw^ ad>nto to )ra for» i^ad tfw ra«igf8 ^wn hetB as 
Um for sMttog sfjM-adMn^istorad Amfs ^m? C^ger to 
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in. HOW WELL HAS 
PENNSYLVANIA'S 
SYSTEM PERFORMED? 

Overview of Pennsyivania^s Day Care 
Monitoring System 

The instrument-based program monitoring 
system devdoi^ by Pennsylvania is known as the 
Child Devdopment Program Evaluation (CDPE). 
The system was implemented in day care craters in 
1978 and has been used continuously unce thai as 
the principal basis for licensing all centers in the 
state. 

Pennsylvania is one of a handful of states that 
requires both private and publicly funded day care 
centers to comply with a single set of state 
raiuirements in order to be licensed. The CDPE 
thus indudes both items that are designed to ensure 
compliance with basic health and safety re- 
quirements (covered under licensing requirements 
in many states) and items that foctis on prognun 
criteria (described under program development in 
many states). 

The CDPE questionnaire measures compliance 
with state regulations in Pennsylvania. The regula- 
tions are very specific with regard to required prac- 
tices and standards and are grouped into seven 
categories: administration, nutrition, social ser- 
vices, transportation, health, child development 
and environmental safety. The complete instru- 



ment, which consists of 279 items that are each 
clearly linked to a panicular regulation, is ad- 
ministered annually to all day care centers. Each of 
the items on the CDPE is assigned a weight based 
on its importance in redudng risk to children. The 
questionnaires are preceded for Msy scoring and 
entry into a computerized data pnmsssing and in- 
formation system, or for manual proce^ing. 

Recently, Pennsylvania has developed a 
shorter version of the CDPE, referred to as an m- 
dicator checklist, which indiKies selected itms 
from the complete questionnaire and can used to 
predict im-formance. The short form that is now 
being tested contains only 18 items. It is antidpated 
that the indicator checklist couid be used on an 
alternating Imsis with the comprehensive inurn- 
ment, that it would be a good predictor of prc^ram 
performance, and that it would reduce monitoring 
costs to the state. Pennsylvania's experience with 
devdoping the indicator diecklist has indicted 
that »milar nnnhodok^es can be applied to reduce 
the Iragth of many din^erent types of state licensing 
and monitcmng questionnaires, while preserving 
the validity of the questionnaire's measures of com- 
pliance and program quality. 

Pennsylvania is particularly advanced in hav- 
ing linked the CDPE system with the state's fman* 
dal and statistical reporting systems. The benefidal 
effects of this linkage have already been described 
in terms of Pennsylvania's ability to make sensitive 
policy decisions and reduce costs based on accurate 
and timely information. 
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An FM can prov^ ra^Sy dv^^^e. 
qusititBllvB informafon m^yz^ 
coa^Amco trends for Mdtora (e.g,. 
roQlons) within a slirto. Ite^^ com* 
pttsnco sct^M from IPM sysfsni 

damonstrBte substantial im- 
provemanls in the qi^My (rf care pro- 
vided by day cara cantem, (See 

Hav^ mg/k^ scom \^ou^ by 
coo^xment area (e.g.. i wfe ninmt i a toi, 
enviwwTentel s^^K me sta^ vfw 
able to target tr^^ m hearth for day 
care center in the Soutt^a^ 
region std In both heemi and ad- 
min ia t r at ton for pro v ideTa In the Cen- 
tral re^^on. 
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Components of the CDPE 

The CDPE system has five Iwic components 
that are described in detail in Volume 2 of this 
series. The componoits are: 

• Pennsylvania's day care regulations; 

• The CDPE questionnaires (long and 
short versions); 

• CDPE OKiing manual; 

• Pennsylvania's system for assigning 
weights to spedfic CDPE items; and 

• forms for reporting CDPE results. 

These componrats are all essential feature of 
the CDPE system. The itons on the que^nnaires 
are, in many cases, simply rqrfirased from par- 
ticular regulations. The coding manual is used to 
score the CDPE responses from a day care crater 
and to ensure consist^y in the way the CDPE is 
administered. The w^ghting system is used to score 
the re3iK)nses; weights reflect imtwrtant policy ran- 
sidoations with r^ard to items <m the CDPE, such 
as those i^rtaining to safety, that cannm be com- 
promised and which are suHident grounds for de* 
nying a day care Ui^se. The reporting fcHim drive 
the system and make monitoring possible by pro- 



viding regular measures for program quality, 
health, and safety at toth the provider and 
resional/state levels. 

Types and Level of Information Generated 

The CDPE system generates three levels of in- 
formation: 

• provido* information; 

• r^onal informatim; and 

• statewide information. 

Basically, the synem fwdw» information 
needed to make all of the monitoring decisions 
about a particular provider. Provider mformation 
includes scores for each of the seven categories of 
regulation covered by the CDPE and a composite 
score on the CDPE It is possible to obtain 
historical trends for the porformame of a particular 
provido^ and to perfcmn analysa of the extent to 
which p^formancc in one category (e.g,, sodal ser- 
vices) is corrdated to i^formance in c^ers (c,g.^ 
child developmoit). 

The CDPE also genmtes summary tn'onna- 
tion on category scores and composite sct rrs by 
region. Comparisons among providers are possible 
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both by type of provider (e.g., non-profit, for- 
profit) and by general statistics for the region (e.g., 
>core ranges, average scores by eatery). 

Statewide category scores and composite 
scores provide perspectives on whether providers 
are improving in general and on the effects of 
changes in regulations and policy. Relative im- 
\ provcments in monitoring efforts in various regions 
i can be observed. .When CDPE results are linked to 
financial and statistical information systems, an 
even broader range of policy questions can be ad- 
dressed. 

Each of these levels and ty|^ of information is 
readily available to the state staff who make u% of 
the information. For example, monitors at the 
regional level may obtain rapid feedback on per- 
formance of particular providers. Regional 
managers have a basis for evaluating the 
performance of providers in their region as com- 
pared with those of other r^ons. Central office 
staff, state budget personnel, and legislators have 
convenient access to general state levels of perfor- 
I mance as these change over time. 

The Implementation Process in 
}^sri]tsyivania 

Use of the CDPE questionnaire began in 1978. 
The implementation process, however, is ongoing, 
with constant improvements being made in the 
basic system. Pennsylvania staff estimate that it 
took roughly 18 months to establish a functioning 
system. The total cost for developing 
Pennsylvania's system has been ^timated at 
$400,000, including the development of data pro- 
casing systems. This development cost could be 
substantially reduced for a state interested in 
transferring Pennsylvania's technology and 
methodology and adapting them to its own re- 
quirements. 

Pennsylvania's cost of monitoring day care us- 
ing the CDPE is estimated at S400,000 per year. 
This figure includes costs related to the state's en- 
tire system, such as staff, travel, data prc^essing, 
report production, and maintenance of the CDPE 
in light of policy and regulation changes. Th^e 
costs will vary substantially for another state, 
depending on the caseload size, the background 
and training of the monitoring staff, frequency of 
monitoring visits, size of the state, travel time lo 
reach provider sites, and,, of course, the number of 
providers in the state. Each of these cost areas re 
quires imponant policy decisions that must be 
made by legislators and administrative decisions 
made by central -of Hce day care managers. 
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These annual monitoring costs represent a sav- 
ing compared to Pennsylvania's costs before 
the CDPE was introduced. Further improvemen 
to the CDPE are likely to reduce these costs even 
more. For example, if the short version of an in- 
strument tike the CDPE is used, additional cost 
savings would occur. 

In addition to the time and costs involved, two 
f^ors war especially critical in Pennsylvania's 
implementation procos: the involvement of pro- 
viders and the level of state commitment to im- 
plementation. From the very beginning of the pro- 
jea it was felt that devdopnmit of the system re- 
quired the involvraient of providers and state, 
regional and cratral office program staff. The in- 
volvement of providers served both to enhance the 
quality of the items on the questionnaire and to 
minimize the suspicion and distrust that are often 
aroused when major administrative and regulatory 
changes are made. Continuous participation of 
providers in designing the questionnaire, assigning 
weights to items, and conducting field trials helped 
to ensure a high level of acceptance when the ques- 
tionnaire was first used for licensing and funding 
purposes. 

Painsylvania's level of commitment to im- 
plementation was high, and the degree of commit- 
ment was a key element in the success of the im- 
plementation process. The substantial costs and the 
long duration of the implementation effort re- 
quired careful planning and execution on the part 
of day care staff and trust on the part of state 
legislators and executives. The establishment of the 
CDPE was clearly not a quick panacea but a com- 
prehensive and thoughtful solution to particular 
concerns thai Pennsylvania faced in its monitoring 
effort. A lower level of state commitment would 
have endangered the entire concept of an IPM 
system. 

Pennsylvania's experience with the CDPE sug- 
gests many of the issues that other slates will need 
to address in implementing an IPM. These issues 
are presented briefly in the next section. 
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IV. HOW CAN I EVALUATE MY 
STATE'S NEED FOR AN IPM 
SYSTEM? 

An instrument-based program monitoring 
sjrstem is a useful tool for {x>licymakers, but not ill 
states need such a system. An interested state may 
follow five steps to assess whether an IPM system is 
appropriate and to initiate the development process 
if the state decides to proceed. These steps are 
outlined below. 

1. Evaluate the State's Social 
Services Environment 

Pennsylvania's motivation for establishing its 
IPM was a response to chan^ in the social services 
environment, such as increased federal and state 
funding of day care, the revision of the Federal In- 
toagency Day Care Requironents, expansion in 
the number of day care facilities, and the esublish- 
ment of a formal organization of day care pro- 
viders. The need for regulatory accountability, the 
large amounts of pubik funds rammitted to day 
care, and a iK}tentiaily larger and more politically 
active group of providers were major concerns to 
Pennsylvania that may shared by other states. 

A sute's view of its role in day care monitoring 
is of fundamental importance in decisions about 
whether to implement IPM systems. States that 
provide little funding for day care have relatively 
less need for state monitoring in any form. The 
IPM approach typically assumes that the state will 
take an active role in program monitoring, in- 
cluding visits to providers in order to assess the 
quality of programs. 

A third set of circumstances that IPM can ad- 
dress is the increasing costs for states that intend to 
take an active role in monitoring. As discussed 
above, making monitoring more consistent and ef- 
ficient may help to reduce both direct costs (e.g., 
cost of monitors* time) and other costs (e.g., costs 
of litigation that may arise as a result of inconsis- 
tent monitoring of providers.) 

To decide whether it needs an IPM system, a 
state will have to examine both its current situation 
and future directions in its approach to monitoring 

2. Review Materials on IPM and the 
Experience of Qther States 

The concept of IPM is new, and the best 
descriptive materials available are a scries of ar- 
ticles that explain how Pennsylvania's system 
works and the underlying framework of child 
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development research on which it is based. Similar- 
ly, Pennsylvania's experience provides the best ex- 
ample of what the IPM approach can accomplish. 
Interested states may wish to arrange for consulta- 
tion with the Pennsylvania staff who have im- 
plemented the concept successfully. 

3. Evaluate Costs and Potential Benefits 
of the IPM Approach 

The costs and toicfits that Pennsylvania has 
realized from its implementation of an IPM have 
been described above* Each state will need to base 
cost and bmefit estimate on the particular design 
of its own piamied syston and on its budget for 
developing new systems. In particular, costs may be 
reduced if a state has easily adaptable regulations, 
positive relationships with provider organizations, 
and well developed computerized information 
systems. 

4. Make a State Commitment to 
Implement the IPM System 

The level of state commitment must be suffi- 
cient to support the implemenution proems 
through potentially difficult periods. Typically, the 
commitment must be made by top officials in the 
state's department that is concern^ with day care 
monitoring (e.g,, l>partment of Public Welfare, 
Department of Human Services, Depanment of 
Education), and supported by legislative bodies 
that are responsible for budget approval. Often, 
coordination is r^uired among several subdepart- 
ment organizations, such as program development, 
management information systems, and operations 
units. 

5. Assess Regulations and 
Legal Requirements 

A successful IPM system must be supported by 
explicit regulations that can be easily translated into 
questions on the monitoring instrument. Some 
work to improve the presentation and sf^ificity of 
regulations may be required for designing the ques- 
tionnaire and implementing the system. 

Similarly, a state must address basic legal and 
policy issues such as: 

• Will the system cover all providers or 
only those who are publicly funded? 

• How will a new system be *grandfathcred' 
or phased in? and 

• Will system scores be used to deny public 
funding to low-scoring providers or serve 
only as a basis for technical assistance 
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and comparisons aroopf providm that 
compete for public funding 

Such legal considerations must be resolved by 
the top policymakers in the staters govemmoit 
before implemratation begins. 

6. Formulate an ImpleiiiMtatton Plan 

The successful imptenoitation of an IPM 
system is largely depemlent on Imving a dtor, wdl 
<Hganized implonentation plan that ircludcs the 
following: 

• dear objectives that specify ^^Amt is to 1^ 
Kcomirfislmit why tl» state is ctevdoinng 
an IPM, and wtet i^Ms are likely to arise 
that will mflueim the devekqmmit effort; 

• dear as^nments of spedflc resiK)nsitH!- 
ities to the individual staff nmnbors who 
will perform the imptenmumion; 

• a Scheie for imfriementatimi that shows 
all of the tasks to be Kcomj^ished, thdr 
sequence for completion, critkal comple- 
tion dates, and the timing of progress 
reports; and 

• tnnigets that cover the allocation of staff 
time and other resources to particular 
implementation tasks* 

When the plan is comi^ete, it should be review- 
ed and approval by all state officials who have con- 
trol over the work to be done and the resources 
needed to accomplish the tasks. The plan should be 
reviewed periodically in the a>urse of implonenta- 
tion, and nec^sary revisions should he made and 
agreed upon by all managers and staff involved. 
By formulating and , adhering to the plan, a 
state has a gr»ter cham» of antidpating potostial 
problems and ensuring a higher degree of satisfac- 
tion with the IPM system that is prtKiuced. 
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Further Information: 

Copies of Consortitun publications, the videotape 
presentation, and additional information on 
instniinait-lnsed program monitoring are available 
from: 

I^. Rkhard Ffeoe 
OMisortium Project Director 
Pennsylvania Office of 
Oiiklren, Youth and Families 
1314 ^forth Second Street 
Harrisburg, Pennslyvania 171^ 

(717) 787-2724 

Ms. G. Hoot 

Project Manage 

Peat, Marwick, Mitchell & Co. 

1990 K Street, N.W. 

Washington, D.C. 20006 

(202) 223-9325 

Ms. MadeUae Dowliag 

Project Offif^ 
OPD, Room 723E 
Hubert Humphrey Building 
200 Independence Ave.. S.W. 
Washington, D.C. 20201 

(^) 245-6233 
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